Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drop Python 3.6 #526

Merged
merged 13 commits into from
Feb 1, 2022
Merged

Drop Python 3.6 #526

merged 13 commits into from
Feb 1, 2022

Conversation

DiddiLeija
Copy link
Collaborator

Closes #525.

Any suggestions to remove 3.6-only code?

@henryiii
Copy link
Collaborator

I'd rather hope my recent changes going in (like the nox file listing) would happen before nox drops 3.6. GHA/Azure's ubuntu-18.04 still runs Python 3.6 if you use pipx, CentOS 7 & 8 both have python3 -> 3.6, etc. So don't see a huge need to push this out immediately.

I'd use pyupgrade to remove 3.6 code, see https://scikit-hep.org/developer/style. I'd also inject from __future__ import annotations using isort then run it, which will clean up annotations.

@DiddiLeija
Copy link
Collaborator Author

DiddiLeija commented Dec 24, 2021

I'd rather hope my recent changes going in (like the nox file listing) would happen before nox drops 3.6. GHA/Azure's ubuntu-18.04 still runs Python 3.6 if you use pipx, CentOS 7 & 8 both have python3 -> 3.6, etc. So don't see a huge need to push this out immediately.

I'd use pyupgrade to remove 3.6 code, see https://scikit-hep.org/developer/style. I'd also inject from __future__ import annotations using isort then run it, which will clean up annotations.

Since this seems to be covered by #530, I'll wait for that PR before merging this.

@DiddiLeija
Copy link
Collaborator Author

Ah, another weird CI failure.

@DiddiLeija DiddiLeija closed this Dec 27, 2021
@DiddiLeija DiddiLeija reopened this Dec 27, 2021
@DiddiLeija DiddiLeija marked this pull request as ready for review December 27, 2021 13:28
@DiddiLeija
Copy link
Collaborator Author

I think our current pre-commit stuff has cleaned the code for dropping Python 3.6, right?

@Tolker-KU
Copy link
Contributor

I think our current pre-commit stuff has cleaned the code for dropping Python 3.6, right?

I think the arg to pyupgrade should be bumped here: https://github.com/theacodes/nox/blob/main/.pre-commit-config.yaml#L29

@cjolowicz
Copy link
Collaborator

cjolowicz commented Dec 27, 2021

As per #526 (comment), we should wait a little before dropping Python 3.6, because it will break leave out people using pipx run nox on Ubuntu 18.04 runners, as well as people installing Nox with the system Python on the current CentOS release. I understand @henryiii 's comment as requesting to cut a Nox release with recently added features before we drop Python 3.6.

As per that same comment, a good way to upgrade the source code to use Python 3.7 features is to add the __future__ annotations import using isort, and to replace --py36-plus with --py37-plus in the pyupgrade hook configuration. Could also happen in a follow-up PR to this one.

@henryiii
Copy link
Collaborator

If you add me to your fork, I'm happy to help. You can use the following to inject the import:

git ls-files '*.py' | xargs pipx run isort -a "from __future__ import annotations"

I also always git grep for "version_info", "Python 3", and a few others things. I've most recently done this with tomerfiliba/plumbum#573.

Dropping 2.7 & 3.5 is huge and really helps with maintenance. I don't think dropping 3.6 will be nearly as impactful, though the annotations are nice.

PS: it will not "break" people using pipx run nox on Ubuntu 18.04, it just will mean they don't get the new features we have been adding, as they will no longer get any nox upgrades once the drop happens. That's why it's nice to make sure there's an up-to-date release.

@theacodes
Copy link
Collaborator

Is there any compelling reason to drop 3.6 right now? Are there any specific things in 3.6 that are holding us back?

@DiddiLeija
Copy link
Collaborator Author

DiddiLeija commented Jan 8, 2022

I don't know about strong reasons to do this. The only reasons I know could be issues with installing dependencies for tests, and that Python 3.6 is EOL'ed.

@henryiii
Copy link
Collaborator

There were lots of benefits to going 3.6+, but there's really not as much going to 3.7+ beyond from __future__ import annotations (which is really nice, but if you've already written Python 3.6 annotations, it's not exactly pressing). Usually core libraries start dropping support ~6 months after EOL (if I remember correctly). Given Requires-Python, this likely won't cause a problem for a while longer, as well - since old versions keep working, so it's only when you want to start requiring new versions of 3.7+ only libraries that you'll likely hit a problem.

3.8 was a bit bigger from an API standpoint, as it lets you write positional only arguments, so designing a new API (like NumPy's new API) benefits from 3.8+. But as far as maintaining an existing library, I don't think there's much of an issue with 3.6+ for a while.

I can help update the PR then you can see if you think it's worth dropping 3.6. :)

@henryiii
Copy link
Collaborator

@DiddiLeija if you want to give me access to your repo I can set pyupgrade & isort to clean up the 3.6 code, or you can modify the pre-commit file as follows:

# isort hook must be before the pyupgrade one, and add the following entry:
    args: ["-a", "from __future__ import annotations"]

# pyupgrade hook should be updated from --py36-plus like this:
    args: ["--py37-plus"]

This should automatically clean up what it can, then it's easy to compare 3.6+ with 3.7+.

@DiddiLeija
Copy link
Collaborator Author

Hi @henryiii, sorry for being offline all this time (I was busy with offline responsibilities). I will try to do it by myself, but if I need help, I will add you to my fork ;)

@DiddiLeija
Copy link
Collaborator Author

Ok... I tried what suggested in #526 (comment). Any comments about the changes?

@henryiii
Copy link
Collaborator

Why did CHANGELOG.md and CONTRIBUTING.md change? Is there an issue with endlines or something on your git setup? That's odd.

@DiddiLeija
Copy link
Collaborator Author

Hmm... I surely made something wrong 😕.

@henryiii, I added you to DiddiLeija/nox.

@henryiii
Copy link
Collaborator

Ahh, those are (now broken) symlinks. Are you on Windows?

@DiddiLeija
Copy link
Collaborator Author

Thanks a lot @henryiii! Well, I suppose this is now ready to merge.

@henryiii
Copy link
Collaborator

I think the point was to compare and see if it's worth dropping now, or waiting a bit. This was you can see the trade offs. A few small workarounds removed, and a little bit nicer syntax, but nothing too major (not like going 3.6+). It's up to the maintainers.

@henryiii
Copy link
Collaborator

(I can rebase this and squash a bit so the automated stuff is operate from the few hand changes if it needs to be kept around for a while)

@DiddiLeija
Copy link
Collaborator Author

(I can rebase this and squash a bit so the automated stuff is operate from the few hand changes if it needs to be kept around for a while)

You can do it if you want, but as far as I know, I will have to squash the commits for the merge ("merge commits" are not allowed here).

@henryiii
Copy link
Collaborator

The idea would be to make it easy to apply in the future if it's not something you want to do right now. By separating the automated parts, you can rerun the automated parts - there weren't a huge number of hand written changes.

(So I'd only do it if you want to wait)

@DiddiLeija
Copy link
Collaborator Author

I get it. From my side, we can merge at any time, but I also would like to hear other maintainers. BTW, a new Nox release is in the air: https://github.com/theacodes/nox/releases/tag/2022.1.7. I don't know where, but we said that we could drop Python 3.6 after that release?

@FollowTheProcess
Copy link
Collaborator

From my side I think it's okay to drop 3.6 in the next release. As @henryiii said it's not as big a win as 3.5 -> 3.6 but as 3.6 is now EOL I think it's fair game for us to drop, we've already cut a release with all the new recent additions (2022.1.7) so 3.6 users won't be missing out on that (which was a good point for not dropping it until after that release)

@DiddiLeija
Copy link
Collaborator Author

Yeah, it's fair (and safe) to merge this now, unless we want to do #561 (comment) first.

@DiddiLeija
Copy link
Collaborator Author

If nobody says "Stop! Don't do this!" before next week, I'll just go ahead and merge this, since seems like we now want to do this.

@henryiii
Copy link
Collaborator

henryiii commented Jan 14, 2022

A few more bits of information:

Python stats from https://mayeut.github.io/manylinux-timeline/ Jan 10 (would be better averaged, but still interesting):

Version Usage
2.7 5.8%
3.5 1.5%
3.6 15.1%
3.7 54.7%
3.8 17.9%
3.9 5.3%
3.10 0.2%

This is based on downloads for 3,564 packages providing manylinux wheels. The full graph is interesting too. This of course is linux users that are using binary packages (but lots of packages have binaries, so a pretty wide sample). I'd expect Windows and macOS users to be much, much more highly biased toward recent versions.

The most interesting thing to me is how wildly popular 3.7 is. I read that 1.8, 2.7, and 3.6 were especially special/popular releases, but I don't know what's up with 3.7. I can think of some major OS's with 3.6, but 3.7 seems well out of proportion; it took the most popular python (from a 2.7/3.6 tie) in middle 2020 and it's still gaining in popularity (from 2.7-3.6 users, I hope!)

I personally don't have a strong opinion either way for a tool like Nox. I'll likely start moving min versions on core stuff around the time Ubuntu 22.04 comes out and is available as a GHA runner, or around the time manylinux drops 3.6. It may depend quite a bit on development - it's easier to develop with a higher min requirement. Less-core stuff will depend on if I'm actively developing it or not - I like to get a benefit from a Python drop if possible. Numerical stuff is supposed to be 3.8+ now, according to NEP 29, so might be doing that too.

@FollowTheProcess
Copy link
Collaborator

@henryiii that data is actually super interesting! Weird how 3.7 is the majority!

@DiddiLeija
Copy link
Collaborator Author

Yeah, why is 3.7 so popular? That's pretty weird for me 🤔

around the time manylinux drops 3.6

Actually, I already decided to ask at pypa/manylinux#1260, maybe that reminder could be helpful.

I like to get a benefit from a Python drop if possible.

Me either. We didn't win a lot with dropping Python 3.6 like dropping 3.5 or 2, I suppose. However, I think it's important to do it. But in the other hand, I'm starting to get concerned by the Ubuntu thing that @henryiii pointed out.

@DiddiLeija
Copy link
Collaborator Author

Ok. Probably, manylinux won't drop 3.6 until 2029. So, I want to know if somebody disagrees with pushing this. If somebody does, I will just close this PR, and recreate it when needed.

cc @theacodes @FollowTheProcess @cjolowicz @dhermes @henryiii

@henryiii
Copy link
Collaborator

henryiii commented Feb 1, 2022

manylinux won't drop 3.6 until 2029

? Pretty sure manylinux will drop 3.6 midway through the year, like always. The main issue I was referring to was ubuntu-18.04 has Python 3.6, so if you use pipx on GHA and use ubuntu-18.04 you get 3.6 (note that ubuntu-latest is ubuntu-20.04, which is fine). Since pip is dropping 3.6, this is becoming a bit nicer. I expect drops to be happening all year.

Note that conda-forge still supports 3.6 (not sure why), so https://github.com/conda-forge/nox-feedstock will need to be updated when this is dropped (https://github.com/conda-forge/pip-feedstock forgot. Both last year and this year!)

To be clear, I don't have any objections either way, I mostly wanted a release with the recent changes out, and that happened. If nice work has gone in supporting an old Python version, it's nice to release it before dropping.

@henryiii
Copy link
Collaborator

henryiii commented Feb 1, 2022

Wow, you are right. Just followed the issue. I'd be shocked if it lasts till 2029, but okay. 3.6 would be like 2.7 at that point...

@henryiii
Copy link
Collaborator

henryiii commented Feb 1, 2022

Ahh, I (we) misread. It says not before mid-year next year (2023 with "standard support" ending for 18.04). That's more reasonable, and inline with past drops.

By the miss-read logic, Python 3.5 would not have been dropped (Ubuntu 16.04, EOL 2026), or Python 3.4 (Ubuntu 14.04, EOL 2024) or Python 2.7 (any OS, including RHEL 8 and Ubuntu 20.04, just not "default" anymore).

@henryiii
Copy link
Collaborator

henryiii commented Feb 1, 2022

3.5 became very expensive to support in cibuildwheel & manylinux (more so than 2.7, actually, due to the ending of patch updates for it). 3.6 might become that way too, but currently, it's not at all hard to support there.

@FollowTheProcess
Copy link
Collaborator

I'm +1 for dropping 👍🏻

@dhermes
Copy link
Collaborator

dhermes commented Feb 1, 2022

I'm big 👍 for dropping. I feel like most projects have a policy of "last 3 major releases" so it would be nice to get rid of 3.7 too (the download numbers ruin this argument).

@henryiii
Copy link
Collaborator

henryiii commented Feb 1, 2022

I think I've seen NEP 29 (42 months) and official Python EOL (5 years), but not "last three" - the release cadence of Python changed around 3.8, so last three would change in size of support window to only 3 years, while it would have been 5.5 before. This is not a PyData project, so I'd not go drop Python versions that are not past official Python EOL. (I rather wish NumPy had adopted Python EOL, one rule would be nicer than several - I understand why, but doesn't mean I love it)

5 years is often "normal" LTS support, too.

@DiddiLeija
Copy link
Collaborator Author

Ok, since it seems like we can drop Python 3.6, I'm gonna merge this. I think we will drop Python 3.7 the next year or even later (I don't really like the "last 3 major releases" policy). Thanks everybody for the guidance!

@DiddiLeija DiddiLeija merged commit a35704f into wntrblm:main Feb 1, 2022
@DiddiLeija DiddiLeija deleted the drop-py36 branch February 1, 2022 22:36
@henryiii
Copy link
Collaborator

henryiii commented Feb 1, 2022

Python 3.7's EOL is Jun 27, 2023.

@henryiii
Copy link
Collaborator

henryiii commented Feb 1, 2022

After that, it will be every October, in sync with new Python releases.

@DiddiLeija
Copy link
Collaborator Author

@henryiii thanks for the information! I didn't know that new Pythons will EOL in October.

@henryiii henryiii mentioned this pull request Feb 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Dropping support for python 3.6 EOL?
7 participants